“The animal will expect a food reward every time it does something right, and will become upset and confused if the food reward does not come.”
Let’s take this apart. Often it’s helpful to invert the criticism to see how a traditional trainer would respond to it.
Does a traditional trainer correct every wrong response?
The answer is yes. Otherwise, the animal becomes upset and confused. Sometimes forging in heel is okay, but other times it gets a collar correction. Sometimes it’s okay to whine on a sit stay, but other times it earns a harsh “No!” (This is one of the reasons vocalization is such a hard problem to fix, regardless of your training paradigm, because the criteria is muddy) Everyone is in agreement that dogs perform better under clear criteria.
So yes, the traditional trainer corrects every failure, and the clicker trainer reinforces every success.
But, just like the traditional trainer has many different forms of correction available, the clicker trainer has many different forms of reinforcement available.
In addition to primary reinforcers like food and play, positively trained cues are reinforcers. I can reward autosits by resuming heeling. I can reinforce a whistle sit by giving my dog a cast to the bird. Stopped contacts are reinforced by continuing the agility run. I can smile at my dog on a long sit, or scratch his butt after the first scent article. Finishing to heel reinforces outing the dumbell reinforces a quick pick up reinforces a fast send. I can reinforce practically anything with a series of nose touches, which I take great pains to make highly reinforcing.
This is not a universal position. A more common answer is putting the animal on a variable reinforcement schedule. Often this is done with pet dogs, where owners are very concerned about getting their animals off treats as soon as possible. Competition trainers are more tolerant of having reinforcement (and punishment) tools in training sessions. However, I feel that while variable reinforcement schedules render the behavior very resistant to extinction, it does not necessarily improve reliability. I feel that goal is better met by a robust rate of reinforcement with a wide variety of reinforcers.
Friday, May 17, 2013
Thursday, July 26, 2012
Is Skinner the Koehler of Clicker?
Among creationists, it is common to seize upon the mistakes in Darwin's work and use those mistakes to cast doubt on the rest of his work.
If he got this wrong, the thinking goes, what else might he have gotten wrong?
At first that sounds like a valid question. There is reasonable doubt, this bears more research.
But creationists are not interested in research. They are already in possession of the truth. They have a prophet. There is no doubt in prophets.
Because they are tied to the idea of a prophet, the prophet's work must be wholly true or wholly false. When that is where you start from, it is easy to see how that kind of thinking is placed on other figures. A creationist cannot understand why an evolutionist might still revere the thinking of a man (by their definition already fallen and untrustworthy) that is shown to be faulty in places.
This same line of thinking can be found in the camps of "balanced trainers" (their label, not mine, and one I am not comfortable with because I cannot figure out what the heck they mean by it) that are fond of attacking "pure positive" strawmen. The most popular strawman is the father of operant conditioning himself, B. F. Skinner.
By all accounts, B. F. Skinner was not a pleasant person. He was egotistical and, worse, his ego was an incredibly delicate thing that required much appeasement and affirmation. He was not even the first, Edward Thorndike's work with cats predates Skinner's pigeons by decades. Skinner's ideas (and politics) were radical and he was vocal about both (and Thorndike was into eugenics). Most notable of these was his concept of "radical behaviorism." Radical behaviorism is the philosophy that all behavior performed by an organism is solely the product of rewards and punishments, none of the behavior originates within the organism itself.
No one today is a radical behaviorist.
No one.
No one is a radial behaviorist in the same way that no one is an Olympian (a worshiper of the Ancient Greek pantheon, not a world-class athlete in London tomorrow).
But all dog training is done under the light of his work.
Dr. Sophia Yin has a superb post on science vs. craft this week.
Dog training, as a profession, is one of apprenticeship more than formal study. There are some trade schools, but they tend not to be highly regarded and are not a replacement for training hours. There is little regulation, anyone can hang out their shingle as a dog trainer. College-education based careers are not common. This promotes a culture of prophets.
Prophets hand down completed works. Completed works are not to be improved upon.
The completed works of prophets are also inscrutable. One will always be a student of the master. If one has a question on the work, surely there is a more senior disciple on hand to explain why the work is infallible and changes are not to be made. If they are, against all advice, they are sure to offer poorer results.
To the clicker trainer, the fallibility of humans is not a problem. We have scientists, not prophets at the head of our cult. Scientists are allowed to be wrong. It is okay, we will do better tomorrow. We are allowed to see farther by standing on the shoulders of giants.
If he got this wrong, the thinking goes, what else might he have gotten wrong?
At first that sounds like a valid question. There is reasonable doubt, this bears more research.
But creationists are not interested in research. They are already in possession of the truth. They have a prophet. There is no doubt in prophets.
Because they are tied to the idea of a prophet, the prophet's work must be wholly true or wholly false. When that is where you start from, it is easy to see how that kind of thinking is placed on other figures. A creationist cannot understand why an evolutionist might still revere the thinking of a man (by their definition already fallen and untrustworthy) that is shown to be faulty in places.
This same line of thinking can be found in the camps of "balanced trainers" (their label, not mine, and one I am not comfortable with because I cannot figure out what the heck they mean by it) that are fond of attacking "pure positive" strawmen. The most popular strawman is the father of operant conditioning himself, B. F. Skinner.
By all accounts, B. F. Skinner was not a pleasant person. He was egotistical and, worse, his ego was an incredibly delicate thing that required much appeasement and affirmation. He was not even the first, Edward Thorndike's work with cats predates Skinner's pigeons by decades. Skinner's ideas (and politics) were radical and he was vocal about both (and Thorndike was into eugenics). Most notable of these was his concept of "radical behaviorism." Radical behaviorism is the philosophy that all behavior performed by an organism is solely the product of rewards and punishments, none of the behavior originates within the organism itself.
No one today is a radical behaviorist.
No one.
No one is a radial behaviorist in the same way that no one is an Olympian (a worshiper of the Ancient Greek pantheon, not a world-class athlete in London tomorrow).
But all dog training is done under the light of his work.
Dr. Sophia Yin has a superb post on science vs. craft this week.
Dog training, as a profession, is one of apprenticeship more than formal study. There are some trade schools, but they tend not to be highly regarded and are not a replacement for training hours. There is little regulation, anyone can hang out their shingle as a dog trainer. College-education based careers are not common. This promotes a culture of prophets.
Prophets hand down completed works. Completed works are not to be improved upon.
The completed works of prophets are also inscrutable. One will always be a student of the master. If one has a question on the work, surely there is a more senior disciple on hand to explain why the work is infallible and changes are not to be made. If they are, against all advice, they are sure to offer poorer results.
To the clicker trainer, the fallibility of humans is not a problem. We have scientists, not prophets at the head of our cult. Scientists are allowed to be wrong. It is okay, we will do better tomorrow. We are allowed to see farther by standing on the shoulders of giants.
Tuesday, March 6, 2012
Adam's Task II: Preface
N.B. All blockquotes are from Vicki Hearne's Adam's Task: Calling Animals by Name unless otherwise specified. My computer based kindle reader isn't working today, so I don't have page numbers. This also means all quotations were hand typed, not copied and pasted, so some minor errors may exist.
Adam's Task was first published in 1986.
The second edition was published in 1994 and included the preface discussed in today's post.
The edition I am reading from was published in 2007. Added to this edition was the introduction by Donald McCaig, which was covered previously.
Vicki Hearne died in 2001.
The opening line of the 1994 preface is
The history of the scientific study of animals, behavior, and cognition is central to understanding the story Hearne is telling. I was very put off when I was first reading the preface, because the attitude of scientists towards animals is very different than the attitude I am used to. I live in an era where canine cognition is studied not only as a model for human cognition, but for it's own interests and merits. There are books like Alexandra Horowitz' Inside of a Dog: What Dogs See, Smell, and Know, Patricia McConnell's The Other End of the Leash and For the Love of a Dog. That Patricia McConnell's profession, ethologist, or the biological study of animal behavior, exsits is a testament to the difference between the scientific understanding of animals in Hearne's time and mine.
So, even if Hearne paints herself as a lone voice speaking up for the animals and aggrandizes her suffering in pursuit of her "eccentric, crank project," I can forgive her. It's her preface and no doubt she faced quite a lot of criticism and roadblocks in her project.
There are a couple of paragraphs about the troubling rise of anti-dog legislation. In this, nothing has changed in the last ten years. Notable is the quote
You see this very often in literature about Pit Bulls. Opponents would have every dog a baby-killer in waiting, supporters wave the banner of the Nanny Dog. It is harder to find a more nuanced account of the breed, but nuance is always hard to come by in emotional issues.
More relevant to this blog and the theme of changing perceptions of animal research is what follows.
Hearne is speaking of veterinary behaviorists (NOT radical Skinner behaviorists, which is another post).
I will take Hearne at her word, that there was "no genuine exchange" between trainers and scientists. Today that is not the case. I would like to imagine Hearne would be pleased with this communication, but I doubt it. I suspect she would feel the scientists are talking to the wrong trainers (namely, trainers who use positive reinforcement exclusively or primarily, and even worse, food) and reaching the wrong conclusions.
A major theme of this series of blog posts is that Hearne says she is seeking to reconcile the academic's and the trainer's understanding of animals, but what she does is condemn the academic for not bowing to the trainer's understanding.
This is a good question!
Aaand Hearne doesn't answer it. Awesome. She does reference the end of "How to Say "Fetch!" so perhaps we will get some explanation when we get to that chapter.
I am not holding my breath.
Now for some snide shots of Hearne-is-a-crappy-writer.
I have read and re-read this sentance sixteen times, and I still have no idea what she is getting at.
She is way, way too fond of asides. I was too. When I was in fourth grade.
Should have said:
"It is something, at nearing fifty, to find myself accompanied in my crank project to find a language to reveal what is crucial to the fact that good trainers' way of talking is counter to the science."
I think. I think she might have left out a noun or two in that sentence somewhere. But that's my best guess at what she's trying to say.
Hearne does this all the time. She introduces a thought with symbolic words like this, "could be dancing instead". In itself, that's not bad. In fact, it's a perfectly valid literary technique. But in order for it to work, you have to explain the idea. You have to break it down, get detailed, show how and why that image works to encapsulate a concept. But Hearne never follows the metaphor through. She just toss it out there and moves on. This is something we will see more and more often the deeper we get into the book.
On the whole, even though I have learned things in the last ten years, I still believe this book.
Adam's Task was first published in 1986.
The second edition was published in 1994 and included the preface discussed in today's post.
The edition I am reading from was published in 2007. Added to this edition was the introduction by Donald McCaig, which was covered previously.
Vicki Hearne died in 2001.
The opening line of the 1994 preface is
In 1993 Time magazine announced that anthropomorphism is no longer a sin, that it's okay now to say that animals think, hope, are puzzled, have expectations, are disappointed, even, for some, make their own little plans in a time scheme of their own.
The history of the scientific study of animals, behavior, and cognition is central to understanding the story Hearne is telling. I was very put off when I was first reading the preface, because the attitude of scientists towards animals is very different than the attitude I am used to. I live in an era where canine cognition is studied not only as a model for human cognition, but for it's own interests and merits. There are books like Alexandra Horowitz' Inside of a Dog: What Dogs See, Smell, and Know, Patricia McConnell's The Other End of the Leash and For the Love of a Dog. That Patricia McConnell's profession, ethologist, or the biological study of animal behavior, exsits is a testament to the difference between the scientific understanding of animals in Hearne's time and mine.
So, even if Hearne paints herself as a lone voice speaking up for the animals and aggrandizes her suffering in pursuit of her "eccentric, crank project," I can forgive her. It's her preface and no doubt she faced quite a lot of criticism and roadblocks in her project.
There are a couple of paragraphs about the troubling rise of anti-dog legislation. In this, nothing has changed in the last ten years. Notable is the quote
...the twin images of the ferocious beast and the gentle, loving, free, or frolicsome creature...
You see this very often in literature about Pit Bulls. Opponents would have every dog a baby-killer in waiting, supporters wave the banner of the Nanny Dog. It is harder to find a more nuanced account of the breed, but nuance is always hard to come by in emotional issues.
More relevant to this blog and the theme of changing perceptions of animal research is what follows.
Disturbing also is the new divorce between training and the "new" behaviorists.... By the "new" behaviorism, I mean that board-certifiable specialty that has appeared in the veterinary profession.
Hearne is speaking of veterinary behaviorists (NOT radical Skinner behaviorists, which is another post).
Since drugs are of one sort or another are often a blessing, a momentary reversal of the Fall, this is not necessarily to be deplored, but it seems something of a shame anyhow, that the knowledge of dogs and of training doesn't - no, can't - make it over the college walls. There is... no genuine exchange between training and the academy.
I will take Hearne at her word, that there was "no genuine exchange" between trainers and scientists. Today that is not the case. I would like to imagine Hearne would be pleased with this communication, but I doubt it. I suspect she would feel the scientists are talking to the wrong trainers (namely, trainers who use positive reinforcement exclusively or primarily, and even worse, food) and reaching the wrong conclusions.
A major theme of this series of blog posts is that Hearne says she is seeking to reconcile the academic's and the trainer's understanding of animals, but what she does is condemn the academic for not bowing to the trainer's understanding.
A major issue in this book is authority. Where does it come from, besides our chimplike impulses? who has the right to command whom?
This is a good question!
We can command, follow, only whom and what we can obey, meaning only whom and what we can hear, respond to.
Aaand Hearne doesn't answer it. Awesome. She does reference the end of "How to Say "Fetch!" so perhaps we will get some explanation when we get to that chapter.
I am not holding my breath.
Now for some snide shots of Hearne-is-a-crappy-writer.
Also, there have been a few wonderful books published...and something that marks a major moment, or discovery, of a possibly grown-up consciousness of animals, John Hollander's anthology, The Naming and Blaming of Cats.
I have read and re-read this sentance sixteen times, and I still have no idea what she is getting at.
She is way, way too fond of asides. I was too. When I was in fourth grade.
It is something, at nearing fifty, to find myself accompanied in what was, when I was groping toward it in the seventies and eighties, an eccentric, crank project - finding a language with which to reveal some of what seemed to me to be so crucial to the fact that good trainers, the ones whose animals are so confident and convincing at their work, are precisely the ones whose ways of talking violate the received precepts of religion and science.
Should have said:
"It is something, at nearing fifty, to find myself accompanied in my crank project to find a language to reveal what is crucial to the fact that good trainers' way of talking is counter to the science."
I think. I think she might have left out a noun or two in that sentence somewhere. But that's my best guess at what she's trying to say.
It's disturbing in part because it means that there are dogs out there on drugs that needn't be, that could be dancing instead.
Hearne does this all the time. She introduces a thought with symbolic words like this, "could be dancing instead". In itself, that's not bad. In fact, it's a perfectly valid literary technique. But in order for it to work, you have to explain the idea. You have to break it down, get detailed, show how and why that image works to encapsulate a concept. But Hearne never follows the metaphor through. She just toss it out there and moves on. This is something we will see more and more often the deeper we get into the book.
Friday, February 10, 2012
Aversive Solutions for the Positively Trained Dog
and why they are ill advised.
It is not just because aversive actions are "out of bounds" for the positive trainer, it is because they lack meaning for the dog.
The German language is a great one for very specialized nouns. Qualzucht is a German word for the purposeful breeding of animals with genetic defects; this has been made illegal by nationwide legislation. While there is a straightforward translation of this word, to truly understand the meaning in context requires that you learn and understand quite a bit of German language, history, current events, and the history and current culture of dog usage and breeding in Germany. Kind of a lot to understand one word.
In traditional dog training, the dog is molded into a sit by pushing down on the rump while lifting the head with leash and collar pressure. Through repetition, the dog learns to release pressure on it's neck by assuming the sit position. When the sit correction is applied (a swift, emphatic jerk straight up on the leash), the dog knows how to escape the pressure. In traditional training, most teaching follows this pattern. The traditionally trained dog is well versed in the application and removal of aversive stimuli.
Positively trained dogs do not understand the application and removal of aversive stimuli. They don't "speak German." Negative reinforcement is not in the vocabulary and syntax they have been taught. It is possible to teach them enough for one specific application to make an impact on a specific behavior, but it's a little like taking a college course in German to understand one word. The return on investment is low.
It is not just because aversive actions are "out of bounds" for the positive trainer, it is because they lack meaning for the dog.
The German language is a great one for very specialized nouns. Qualzucht is a German word for the purposeful breeding of animals with genetic defects; this has been made illegal by nationwide legislation. While there is a straightforward translation of this word, to truly understand the meaning in context requires that you learn and understand quite a bit of German language, history, current events, and the history and current culture of dog usage and breeding in Germany. Kind of a lot to understand one word.
In traditional dog training, the dog is molded into a sit by pushing down on the rump while lifting the head with leash and collar pressure. Through repetition, the dog learns to release pressure on it's neck by assuming the sit position. When the sit correction is applied (a swift, emphatic jerk straight up on the leash), the dog knows how to escape the pressure. In traditional training, most teaching follows this pattern. The traditionally trained dog is well versed in the application and removal of aversive stimuli.
Positively trained dogs do not understand the application and removal of aversive stimuli. They don't "speak German." Negative reinforcement is not in the vocabulary and syntax they have been taught. It is possible to teach them enough for one specific application to make an impact on a specific behavior, but it's a little like taking a college course in German to understand one word. The return on investment is low.
Thursday, January 26, 2012
Training with a Clicker vs Clicker Training
I'm confused about this, because I thought that the click was supposed to become a conditioned reinforcer and that you always "mark" when the dog did what you wanted, but fade the treats down to occasional and random (as well as giving other rewards like praise and scritches) so that the dog doesn't only perform when they're hungry. I thought unpredictable reinforcers were actually better at getting a behavior than predictable ones, once the dog has actually learned the behavior.That is a very common misinterpretation. It does work, at least for a while, but you usually end up having to recharge the clicker pretty often. I consider it an inefficient use of the tool.
(The trainers I've worked with used verbal markers rather than clicks. I don't know if that makes a difference in terms of when you need to treat.)
KellyK, Lehrhund: Dog Training Law
Instead of fading rewards, think about raising criteria.
Example: If your dog can do a five minute down stay, you aren't going to click 30 second stays (unless you're fixing something). When a student is in an algebra class, they don't spend time doing basic subtraction except in the context of a harder problem (like every 5 minute stay includes a 30 second stay). If your student CAN'T do a 30 second stay or a basic subtraction problem, they have no business doing advanced work when they don't have the foundation to be successful.
Clickers are really most useful in behavior acquisition. See shaping, where the goal is crystal clear communication. By the time I've put a name on a finished behavior, I usually do not need to be so precise about what I'm reinforcing, so I switch to a verbal mark. At that point, it's a matter of context, timing, and ability to reinforce that determines if it is worthwhile to mark.
Example: You are in training and working on down on recall. Let's say your dog knows this exercise, and does it with confidence. In other words, it's just practice, you aren't fixing anything. When he downs, you might offer a word of praise ("Gooooooood dog") before finishing the recall. It's not appropriate to mark in that situation, because you don't want to end the exercise, but it is appropriate to let the dog know he performed the criteria.
When he fronts and finishes, you can mark and reward, or can release, or you can move into the next exercise. It depends on how the dog was trained and what he needs. Both of my dogs have a tendency to bypass front and auto-finish, so I rarely finish them in practice and often mark and reward the front.
Another note about rewards: I prefer to have a wide range of reinforcers that I can use very often than one reinforcer used sparingly. Developing life rewards and self control (sit to go out the door, heel past the squirrel to chase the squirrel, give up the treat to get the treat) are the key to clicker trained behaviors being useful in real life.
You are right that unpredictable rewards create stronger behavior than predictable ones (compare buying a soda from a vending machine vs playing slots at a casino) but many people get so hyper focused on getting off food rewards that they miss the bigger picture of using rewards in training. Clicker training uses a LOT of food, particularly early in training, and I think people see that and think they're going to have to keep up that rate of reinforcement for the life of the dog. You won't, I promise, even though it looks that way from the start.
Wednesday, January 18, 2012
Adam's Task I: "Pellucid Prose"
I started reading Adam's Task on my Kindle. When you first open a book on the Kindle, it doesn't take you to the cover or table of contents, the first page you see is the first page of writing. So it wasn't until I finished the prologue that I learned it was written by Border Collie trainer and trialer Donald McCaig.
I thought he was a woman.
Take that as you will, but McCaig's prologue sets the tone for the book: sentimentally flowery where passive syntax masquerades as arcane truth.
"Let me backtrack to 1986. I was and am a sheepdog trainer. I believe that training any dog to anything like his full capacity is an intricate, heartfelt, deeply intellectual undertaking which deepens the trainer's soul as surely as it satisfies the dog's. The conversation between trainer and dog is so subtle, dense, and satisfying that I have known great trainers whose ordinary human speech has atrophied. These brilliant linguists cannot explain what they do, and often cannot answer novice's questions because asking that particular question means the questioner can't understand a true answer."Vicki Hearne. Adam's Task: Calling Animals by Name (p. ix). Kindle Edition.
Emphasis mine. I am a deeply theoretical person (hence, the blog's title, which translates literally to teachdog, and poetically to theoryhound) and cannot do anything without understanding how it works, why it works, what to do if it doesn't work, and when to know that it did work. As my understanding of dogs and dog training grows, I am beginning to understand the bold more fully. I can explain myself all day long, but if the listener is not ready for the knowledge, they will not hear it.
My objection is when this is used as an excuse not to even try explaining. This idea will eventually be expanded on in the main text, reasonable reasoning used as an excuse for objectionable action. If you cannot understand a true answer, you do not get an answer.
A blog I read frequently posts short quotes from a variety of sources on dogs, training, learning, and any other topic that touches on some truth related to these (as everything is dog training, this encompasses quite a lot). While I usually agree with the quotes, they clearly mean more to the author than they do to me. Sometimes understanding is beyond our current knowledge, but without being exposed to them our understanding can never grow.
"Dog fanciers (dog show people) whose arcane lingo obscures and excludes these sparks found Vicki's pellucid prose "difficult" and her democratic spirit profoundly unsettling."Vicki Hearne. Adam's Task: Calling Animals by Name (p. x). Kindle Edition.
If you are using the word pellucid, you are not. I suspect McCaig was too allured by alliteration. As we will see, Hearne is anything but "pellucid" and even less democratic.
As I write this, I'm training two Border Collies for sheepdog trials and starting a three-year-old who has been a difficult family pet because her heart is too great for petdom.Vicki Hearne. Adam's Task: Calling Animals by Name (p. xi). Kindle Edition.
This is the first slight against pet owners. This is not a book with a "democratic spirit," this is a book that ridicules and puts down anyone less enlightened than the author.
Thursday, December 15, 2011
On Dialouge
Wouldn’t it be better to say, “In my experience, positive training does not work.”? That phrase opens up a place for dialogue and the possibility that you may be confronted with evidence, which might, over time, allow you to change your perspective and try something new.
Denise Fenzi, What is Possible
This is what this blog is about.
Subscribe to:
Posts (Atom)